While software systems continue to grow in size and complexity, business demands continue to require shorter development cycles. This has led software developers to compromise on functionality, time to market, and quality of software products. Furthermore, the increased schedule pressures and limited availability of resources and skilled labor can lead to problems such as incomplete design of software products, inefficient testing, poor quality, high development and maintenance costs, and the like. This may lead to poor customer satisfaction and a loss of market share for companies developing software.
To improve product quality, many organizations devote an increasing share of their resources to testing and identifying problem areas related to software and the process of software development. Accordingly, it is not unusual to include a quality assurance team in software development projects to identify defects in the software product during and after development of a software product. By identifying and resolving defects before marketing the product to customers, software developers can assure customers of the reliability of their products, and reduce the occurrence of post-sale software fixes such as patches and upgrades which may frustrate their customers.
Software testing may involve verifying the correctness, completeness, security, quality, etc. of a product. During testing, a technical investigation may be performed by, for example, executing a program or application with the intent to find errors. If errors are found, one or more areas in the software code may be identified based on the errors. Therefore, developers may alter the code in the identified regions to obviate the error.
After a defect has been fixed, data regarding the defect, and the resolution of the defect, may be stored in a database. The defects may be classified and analyzed as a whole using, for example, Orthogonal Defect Classification (ODC) and/or a defect analysis starter/defect reduction method (DAS/DRM), which is described in U.S. Patent Application Publication No. 2006/0265188, U.S. Patent Application Publication No. 2006/0251073, and U.S. Patent Application Publication No. 2007/0174023, the contents of each of which are hereby incorporated by reference herein in their entirety. ODC is a commonly used complex quality assessment schema for understanding code related defects uncovered during testing.
It is widely accepted in the testing industry that the least expensive defects to fix are those found earliest in the life cycle. However, a problem in complex system integration testing is that there may be very few comprehensive opportunities for projects to remove defects cost effectively prior to late phase testing, and by that point in the life cycle (i.e., late phase testing) defects are relatively expensive to fix. Furthermore, for many projects there are particular kinds of high impact exposures, e.g., defects in the area of security, that are critical to find and fix, but are also difficult to test.
There are numerous automated code inspection tools available on the market today designed to address this problem; however, for many projects, it is not cost effective for an organization to purchase licenses for all of the tools needed to cover all of the exposures of interest to them. Moreover, even if it was cost effective for an organization to purchase licenses for all of the tools needed to cover all of the exposures, there is no way to understand the return on this investment in terms of the impact on reducing the numbers of defects found in late phase testing and in production.
As a result of these impracticalities, few complex system integration projects avail themselves of automated code inspection defect removal strategies, even though applying them to unit tested code prior to beginning system testing is one of the most cost effective options available. This problem has been addressed in part by, e.g., a service provider assembling a set of code inspection tools designed to address four areas, as shown in TABLE 1 below.
TABLE 1DynamicTechnologiesStatic CodeCodeTypes of analysis:Functional Outputssupportedanalysisanalysis1Industry and BestMaintainability,COBOL, C++,XPractice StandardsRobustness,J2EE/JAVA,ComplianceQuality,ABAP,Changeability,Microsoft.NETPerformance,ProgrammingPractices,ArchitecturalDesign,Documentation2SecurityApplicationWebXPrivacy,ApplicationsAuthentication,Authorization,Client-sideAttacks,CommandExecution,InformationDisclosure,Location, LogicalAttacks3MemoryMemory leaks,WebXManagementMemory accessApplicationserrors, Memorystate tracking,Quantify forapplicationperformanceprofiling,Coverage4Usability andAccessibilityWebXAccessibilityApplications
With this approach, for example, a project (e.g., a software project of an organization) can purchase code inspection services from the service provider on an as-needed basis without requiring any tool purchase or licensing costs for tools they may only need to leverage on a limited basis. Thus, a project may, for example, utilize a plurality of code inspection services (e.g., specifically tailored for their project) and receive code inspection services reports from the service provider. By assembling a set of code inspection tools and providing for purchase of code inspection services on an as-needed basis, utilization of these code inspection services is rendered more cost effective.
However, no defect analysis schema capable of accurately measuring value received from performing specific automated code inspection activities is known to exist. Thus, there is no way to understand the return on this investment (e.g., the purchase of code inspection services) in terms of the impact on reducing the numbers of defects found in late phase testing and in production. That is, the code inspection services reports (for example, from the plurality of code inspection services, e.g., specifically tailored for their project) do not interpret defects uncovered via the automated code inspection subscription service. Rather, such code inspection service reports, for example, only identify defects uncovered via the automated code inspection subscription service. Thus, this automated code inspection subscription service does not allow projects to accurately assess the impact of automated code inspections on, for example, critical exposure areas and does not allow for effective planning of, for example, late phase testing and production support needs.
Accordingly, there exists a need in the art to overcome the deficiencies and limitations described hereinabove.